ENCODER-DECODER FRAMEWORK

Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation (N18-1)

Challenge:	Plane geometry problem solving has gained significant attention as a benchmark to assess the multi-modal reasoning capabilities of large vision-language models.
Approach:	They present a systematic review of existing work in PGPS and summarize their results.
Outcome:	The proposed frameworks are compared with existing frameworks and analyze them according to their architectural designs.

Copied to clipboard

Shuming Ma, Xu Sun, Wei Li, Sujian Li, Wenjie Li, Xuancheng Ren

Marian: Fast Neural Machine Translation in C++ (P18-4)

Challenge:	Existing models tend to memorize words instead of learning meaning of words . existing models tend not to model semantic information, resulting in incorrect sentences .
Approach:	They propose a novel model that generates words by querying distributed word representations . they evaluate model on two paraphrase-oriented tasks, namely text simplification and short abstractive summarization .
Outcome:	The proposed model outperforms the baseline model on two paraphrase-oriented tasks . it achieves state-of-the-art performance on these benchmark datasets .

Copied to clipboard

Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, Alexandra Birch

Machine Translation

Challenge:	In this paper, we present Marian, an efficient and self-contained Neural Machine Translation framework . Marian is written in pure C++ with minimal dependencies .
Approach:	They present Marian, an efficient and self-contained Neural Machine Translation framework written in pure C++ with minimal dependencies.
Outcome:	The proposed framework achieves high training and translation speed with minimal dependencies . it is currently being deployed in multiple European projects .

A Unified Encoder-Decoder Framework with Entity Memory (2022.emnlp-main)

Copied to clipboard

Zhihan Zhang, Wenhao Yu, Chenguang Zhu, Meng Jiang

CGF: Constrained Generation Framework for Query Rewriting in Conversational AI (2022.emnlp-industry)

Challenge:	Existing approaches to index, retrieve, and read documents as evidence suffer from large computational overheads.
Approach:	They propose an encoder-decoder framework with an entity memory that stores entity knowledge as latent representations and pre-trained on Wikipedia along with encoder parameters.
Outcome:	The proposed framework outperforms memory-based and non-memory encoder-decoder models on various entity-intensive question answering and generation tasks.

Copied to clipboard

Jie Hao, Yang Liu, Xing Fan, Saurabh Gupta, Saleh Soltan, Rakesh Chada, Pradeep Natarajan, Chenlei Guo, Gokhan Tur

Paraphrasing

Challenge:	Large-scale conversational AI agents such as Alexa, Siri and Google Assistant help millions of users to perform a lot of tasks.
Approach:	They propose a Constrained Generation Framework for query rewriting at global and personalized levels.
Outcome:	The proposed framework significantly boosts the query rewriting performance.

Neural Open Information Extraction (P18-2)

Copied to clipboard

Lei Cui, Furu Wei, Ming Zhou

Information Extraction & Text Mining

Challenge:	Existing Open IE systems are built on hand-crafted patterns from syntactic parsing, yet they face errors in propagation and compounding at each stage.
Approach:	They propose a neural Open IE approach with an encoder-decoder framework . they propose to learn highly confident arguments and relation tuples bootstrapped from a state-of-the-art Open ie system.
Outcome:	The proposed approach outperforms baseline methods while maintaining comparable computational efficiency.

A Semantically Consistent and Syntactically Variational Encoder-Decoder Framework for Paraphrase Generation (2020.coling-main)

Copied to clipboard

Wenqing Chen, Jidong Tian, Liqiang Xiao, Hao He, Yaohui Jin

Refining Source Representations with Relation Networks for Neural Machine Translation (C18-1)

Challenge:	Paraphrase generation is a longstanding problem in natural language processing (NLP) Neural network-based methods have shown great progress on paraphrase generation.
Approach:	They propose a framework that integrates variational inference on a target-related latent variable to introduce the diversity.
Outcome:	The proposed framework outperforms baseline models on the metrics based on n-gram matching and semantic similarity, and it can generate multiple different paraphrases by assembling different syntactic variables.

Copied to clipboard

Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu

Personalized Review Generation By Expanding Phrases and Attending on Aspect-Aware Representations (P18-2)

Challenge:	Existing neural machine translation frameworks that forget distant information and disregard relationship between source and target words are not effective.
Approach:	They propose to use relation networks to learn better representations of the source . they propose to associate source words with each other to help retain their relationships .
Outcome:	Experiments show that the proposed approach outperforms the encoder-decoder framework on several datasets.

Copied to clipboard

Jianmo Ni, Julian McAuley

Dense Information Flow for Neural Machine Translation (N18-1)

Challenge:	Existing systems that use user and item identity as inputs for review generation are lacking in the field of natural language processing.
Approach:	They propose an encoder-decoder framework that generates personalized reviews by expanding short phrases provided as input to the system.
Outcome:	The proposed model learns representations capable of generating coherent and diverse reviews.

Copied to clipboard

Yanyao Shen, Xu Tan, Di He, Tao Qin, Tie-Yan Liu

Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoder (2022.coling-1)

Challenge:	Recent advances in deep neural networks have improved learning performance for NMT . Residual connections allow features from previous layers to be accumulated to the next layer easily.
Approach:	They propose a densely connected NMT architecture that can train more efficiently for NMT.
Outcome:	The proposed architecture improves learning performance and attention quality on multiple datasets.

Copied to clipboard

Fangyu Lei, Shizhu He, Xiang Li, Jun Zhao, Kang Liu

Reasoning

Challenge:	Existing methods for numerical reasoning are not flexible enough to handle diverse expressions.
Approach:	They propose a Relational Graph enhanced Hybrid table-text Numerical reasoning model with Tree decoder which captures relationship between numerical value, table schema, and text information on the encoder side.
Outcome:	The proposed model outperforms the baseline model and achieves state-of-the-art results on the publicly available tabletext hybrid QA benchmark.

GTR-LSTM: A Triple Encoder for Sentence Generation from RDF Data (P18-1)

Copied to clipboard

Bayu Distiawan Trisedya, Jianzhong Qi, Rui Zhang, Wei Wang

Unsupervised Context Rewriting for Open Domain Conversation (D19-1)

Challenge:	Knowledge bases are becoming an enabling resource for many applications including Q&A systems, recommender systems, and summarization tools.
Approach:	They propose a system to translate RDF triples into natural sentences using an encoder-decoder framework.
Outcome:	The proposed model outperforms the baseline model by 17.6%, 6.0%, and 16.4% in terms of BLEU, METEOR, and TER scores.

Copied to clipboard

Kun Zhou, Kai Zhang, Yu Wu, Shujie Liu, Jingsong Yu

Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation (P19-1)

Challenge:	Existing approaches to model conversation context have drawbacks, such as lack of coreferences and long dependency.
Approach:	They propose a context rewriting method which explicitly rewrites the last utterance by considering context history.
Outcome:	The proposed method outperforms baselines in terms of rewriting quality, multi-turn response generation, and end-to-end retrieval-based chatbots.

Copied to clipboard

Shuming Ma, Pengcheng Yang, Tianyu Liu, Peng Li, Jie Zhou, Xu Sun

A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning (2020.findings-emnlp)

Challenge:	Existing methods for table-to-text generation use encoder-decoder framework, but lack of large parallel data is a problem for many domains.
Approach:	They propose a model to separate table-to-text generation into two stages: key fact prediction and surface realization.
Outcome:	The proposed model achieves 27.34 BLEU score with only 1,000 parallel data, while the baseline model only achieves 9.71 BLUE score.

Copied to clipboard

Deepak Gupta, Asif Ekbal, Pushpak Bhattacharyya

CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)

Challenge:	Existing methods to train neural network-based models for code-mixing are limited due to language specificity of code-mixed text.
Approach:	They propose a deep learning approach to generate code-mixed text from English to multiple languages without any parallel data.
Outcome:	The proposed approach generates a code-mixed text from English to multiple languages without any parallel data.

Copied to clipboard

Zhen Yang, Bojie Hu, Ambyera Han, Shen Huang, Qi Ju

Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation (2021.acl-long)

Challenge:	Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT .
Approach:	They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language.
Outcome:	The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora.

Copied to clipboard

Yang Feng, Shuhao Gu, Dengji Guo, Zhengxin Yang, Chenze Shao

Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning (P18-1)

Challenge:	Neural machine translation models are usually based on attention-based encoder-decoder frameworks.
Approach:	They introduce a seer decoder into the encoder-decoder framework during training . they force the conventional decoded decodes to simulate the behavior of the seer .
Outcome:	The proposed method outperforms baselines on Chinese, English and German translation tasks.

Copied to clipboard

Hongge Chen, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Cho-Jui Hsieh

Multimodality

Challenge:	Visual language grounding is widely studied in modern neural image captioning systems . a novel algorithm for crafting adversarial examples in image captions is proposed .
Approach:	They propose an algorithm to craft adversarial examples in machine vision and perception . their approach provides two evaluation approaches to check if they can mislead systems .
Outcome:	The proposed algorithm can craft visually-similar adversarial examples with randomly targeted captions or keywords, and the results are transferable to other image captioning systems.

Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems (N19-1)

Copied to clipboard

Ting-Rui Chiang, Yun-Nung Chen

Reasoning

Challenge:	Existing methods to solve math word problems require accurate natural language understanding to bridge texts and math expressions.
Approach:	They propose a neural approach to automatically solve math word problems by operating symbols according to their semantic meanings in texts.
Outcome:	The proposed model outperforms state-of-the-art models and the best non-retrieval-based models over 10% accuracy in a Math23K dataset.

Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems (2020.emnlp-main)

Copied to clipboard

Jinghui Qin, Lihui Lin, Xiaodan Liang, Rumin Zhang, Liang Lin

Numerical Reasoning

Challenge:	Existing models focus on one-unknown linear MWPs.
Approach:	They propose a universal expression tree-structured solver that integrates multiple expression trees underlying a MWP into a single expression tree.
Outcome:	The proposed method outperforms state-of-the-art models on a MWPs dataset and generates a universal expression tree explicitly by deciding which symbol to generate .

Improving Domain Adaptation Translation with Domain Invariant and Specific Information (N19-1)

Copied to clipboard

Shuhao Gu, Yang Feng, Qun Liu

Responsible & Trustworthy NLP

Challenge:	Neural machine translation models are based on the encoder-decoder architecture, which makes them overfitting to frequent observations.
Approach:	They propose a method to explicitly model out-of-domain information in an encoder-decoder framework . they propose combining out- of-domain training data with out-out-of domain data .
Outcome:	The proposed method outperforms baselines on multiple data sets.

Graph Enhanced Contrastive Learning for Radiology Findings Summarization (2022.acl-long)

Copied to clipboard

Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, Tsung-Hui Chang

Structural Supervision for Word Alignment and Machine Translation (2022.findings-acl)

Challenge:	Existing methods for automating impression generation have limited the relationship between extra knowledge and the original findings.
Approach:	They propose a framework for automating impression generation that exploits extra knowledge and original findings . they propose combining key words and their relations to extract critical information .
Outcome:	The proposed framework exploits extra knowledge and the original findings in an integrated way . the state-of-the-art results on two datasets confirm the effectiveness of the proposed method .

Copied to clipboard

Lei Li, Kai Fan, Hongjia Li, Chun Yuan

Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation (D18-1)

Challenge:	Existing knowledge on syntactic structure neglects the rich structural information from target tokens and the structural similarity between the source and target sentences.
Approach:	They propose to incorporate syntactic structure of both source and target tokens into the encoder-decoder framework, tightly correlating the internal logic of word alignment and machine translation for multi-task learning.
Outcome:	The proposed method outperforms baselines on four publicly available language pairs and consistently outperformed baselines in alignment accuracy and translation quality.

Copied to clipboard

Junyang Lin, Xu Sun, Xuancheng Ren, Muyu Li, Qi Su

Machine Translation

Challenge:	Neural Machine Translation models treat decoding at each time step equally with the same matrix . conventional methods treat decoder outputs at all time steps with the identical weight matrix causing inaccuracy .
Approach:	They propose a model with a mechanism to control the softness of attention by means of an attention temperature.
Outcome:	The proposed model outperforms baseline models on Chinese-English and English-Vietnamese translations.

Retrieval-Enhanced Adversarial Training for Neural Response Generation (P19-1)

Copied to clipboard

Qingfu Zhu, Lei Cui, Wei-Nan Zhang, Furu Wei, Ting Liu

AdaNSP: Uncertainty-driven Adaptive Decoding in Neural Semantic Parsing (P19-1)

Challenge:	Existing approaches to dialogue systems are labor-intensive and difficult to scale up.
Approach:	They propose a Retrieval-Enhanced Adversarial Training method for neural response generation that leverages an adversarial training paradigm while taking advantage of N-best response candidates from a retrieval-based system to construct the discriminator.
Outcome:	The proposed method outperforms the vanilla Seq2Seq model and conventional adversarial training approach on a large scale dataset.

Copied to clipboard

Xiang Zhang, Shizhu He, Kang Liu, Jun Zhao

Answer-driven Deep Question Generation based on Reinforcement Learning (2020.coling-main)

Challenge:	Semantic parsing (SP) maps a natural language utterance into a formal language . standard Seq2Seq models ignore underlying grammars and may give ill-formed results.
Approach:	They propose an end-to-end model for semantic parsing that transduces a natural language sentence to the formal semantic representation.
Outcome:	The proposed model outperforms the state-of-the-art models and does not need expertise like predefined grammar or sketches in the meantime.

Copied to clipboard

Liuyin Wang, Zihan Xu, Zibo Lin, Haitao Zheng, Ying Shen

Cross-modal Memory Networks for Radiology Report Generation (2021.acl-long)

Challenge:	Existing methods for deep question generation focus on enhancing document representations, but little attention is paid to the answer information.
Approach:	They propose a deep question generation model that makes better use of the target answer as a guidance to facilitate question generation.
Outcome:	The proposed model outperforms state-of-the-art models in automatic and human evaluations on the hotpotQA dataset.

Copied to clipboard

Zhihong Chen, Yaling Shen, Yan Song, Xiang Wan

Multimodality

Challenge:	Medical imaging reports are essential in clinical practice, and generating the reports is beneficial to reduce the burden of radiologists.
Approach:	They propose to use a shared memory to enhance the encoder-decoder framework for radiology report generation.
Outcome:	The proposed model can generate more accurate reports on two widely used datasets.

Learning Universal Sentence Representations with Mean-Max Attention Autoencoder (D18-1)

Copied to clipboard

Minghua Zhang, Yunfang Wu, Weikang Li, Wei Li

UniXcoder: Unified Cross-Modal Pre-training for Code Representation (2022.acl-long)

Challenge:	Existing methods to learn universal sentence representations focus on supervised learning.
Approach:	They propose a mean-max attention autoencoder that uses a multi-head mechanism to reconstruct the input sequence.
Outcome:	The proposed model outperforms state-of-the-art unsupervised single methods on a wide range of 10 transfer tasks.

Copied to clipboard

Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, Jian Yin

Cross-Lingual Unsupervised Sentiment Classification with Multi-View Transfer Learning (2020.acl-main)

Challenge:	Pre-trained models for programming languages have demonstrated great success on code intelligence . however, such pre-tried models are sub-optimal for auto-regressive tasks .
Approach:	They propose a unified cross-modal pre-trained model for programming language that leverages cross-module contents like AST and code comment to enhance code representation.
Outcome:	The proposed model achieves state-of-the-art on most code-related tasks and compares with existing models on zero-shot code-to-code search.

Copied to clipboard

Hongliang Fei, Ping Li

Information Retrieval

Challenge:	Recent neural network models have achieved impressive performance on sentiment classification in English and other languages.
Approach:	They propose an unsupervised sentiment classification model that leverages an uncontrolled machine translation system and a language discriminator to learn a shared representation.
Outcome:	The proposed model outperforms other models on five language pairs.

Improving Neural Machine Translation with Soft Template Prediction (2020.acl-main)

Copied to clipboard

Jian Yang, Shuming Ma, Dongdong Zhang, Zhoujun Li, Ming Zhou

Graph-to-Text Generation with Dynamic Structure Pruning (2022.coling-1)

Challenge:	Recent advances in neural machine translation (NMT) depend on source text to generate translation.
Approach:	They propose to use extracted templates from tree structures as soft target templates to guide the translation procedure.
Outcome:	The proposed model outperforms baseline models on four benchmarks and demonstrates the effectiveness of soft target templates.

Copied to clipboard

Liang Li, Ruiying Geng, Bowen Li, Can Ma, Yinliang Yue, Binhua Li, Yongbin Li

Structured Data in NLP

Challenge:	Recent studies show that explicitly modeling the input graph structure can significantly improve the performance.
Approach:	They propose a structure-aware cross-attention mechanism to re-encode the graph representation conditioning on the newly generated context at each decoding step.
Outcome:	The proposed model improves performance on two graph-to-text datasets with only minor increase on computational cost.

Enhancing Large Language Models for Document-Level Translation Post-Editing Using Monolingual Data (2025.coling-main)

Copied to clipboard

Zongyao Li, Zhiqiang Rao, Hengchao Shang, Jiaxin Guo, Shaojun Li, Daimeng Wei, Hao Yang

tagE: Enabling an Embodied Agent to Understand Human Instructions (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) have excellent performance in many tasks, but they still face challenges in document translation.
Approach:	They propose a method that leverages the capabilities of Large Language Models to optimize document translation using only monolingual data.
Outcome:	The proposed method improves translation quality and improves contextual consistency in document translation using only monolingual data.

Copied to clipboard

Chayan Sarkar, Avik Mitra, Pradip Pramanick, Tapas Nayak

Challenge:	Existing systems for natural language understanding (NLU) are limited due to the inherent ambiguity and incompleteness inherent in natural language.
Approach:	They propose a system to extract tasks from natural language instructions and map them to robots' established collection of skills.
Outcome:	The proposed system outperforms baseline models in the training and evaluation of a dataset featuring complex instructions.

Conversational Semantic Parsing for Dialog State Tracking (2020.emnlp-main)

Copied to clipboard

Jianpeng Cheng, Devang Agrawal, Héctor Martínez Alonso, Shruti Bhargava, Joris Driesen, Federico Flego, Dain Kaplan, Dimitri Kartsaklis, Lin Li, Dhivya Piraviperumal, Jason D. Williams, Hong Yu, Diarmuid Ó Séaghdha, Anders Johannsen

Dialogue Systems & Conversational Agents

Challenge:	Language understanding for task-based dialog systems is often termed "dialog state tracking" (DST) whereas semantic parsing is the task of converting a single-turn utterance to a graphstructured meaning representation, DST is more complex.
Approach:	They propose a framework for dialog state tracking that incorporates semantic compositionality, cross-domain knowledge sharing and co-reference.
Outcome:	The proposed framework improves on state-of-the-art approaches for dialog state tracking (DST) it incorporates semantic compositionality, cross-domain knowledge sharing and co-reference.

Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding (2024.findings-acl)

Copied to clipboard

Jiali Zeng, Fandong Meng, Yongjing Yin, Jie Zhou

Exploring the Impact of Vision Features in News Image Captioning (2023.findings-acl)

Challenge:	Commercial machine translation engines are proficient in addressing the majority of translation requirements.
Approach:	They propose to combine NMT and MT-oriented LLMs to achieve superior translation quality by combining their strengths.
Outcome:	The proposed model can handle complex scenarios beyond the capability of NMT alone.

Copied to clipboard

Junzhe Zhang, Xiaojun Wan

Multimodality

Challenge:	Recent state-of-art models can achieve competitive performance even without vision features.
Approach:	They conduct extensive experiments with mainstream news image captioning models to determine whether vision features contribute to the generation of captions.
Outcome:	The proposed models can achieve competitive performance even without vision features.

Linguistically-Controlled Paraphrase Generation (2025.findings-emnlp)

Copied to clipboard

Mohamed Elgaar, Hadi Amiri

Reduce Redundancy Then Rerank: Enhancing Code Summarization with a Novel Pipeline Framework (2024.lrec-main)

Challenge:	Controlled paraphrase generation produces paraphrases that preserve meaning while allowing precise control over linguistic attributes of output.
Approach:	They introduce an encoder-decoder framework that enables fine-grained control over 40 linguistic attributes in English.
Outcome:	The proposed framework reduces attribute error by up to 34% over existing models .

Copied to clipboard

Xiaoyu Hu, Xu Zhang, Zexu Lin, Deyu Zhou